Foreground and background text in retrieval

Our hypothesis is that certain clauses have foreground functions in text, while other clauses have background functions and that these functions are expressed or reflected in the syntactic structure of the clause. Presumably these clauses will have differing utility for automatic approaches to text understanding; a summarization system might want to utilize background clauses to capture commonalities between numbers of documents while an indexing system might use foreground clauses in order to capture specific characteristics of a certain document. Topic in text for information access This paper gives a short description of a series of experiments we have performed to test our hypotheses that clauses have different functions in transmitting the information flow of text, namely the functions often called topicality or thematic structure. The application area we chose to evaluate our hypotheses through is that of analysis of texts for the purposes of information retrieval. Topicality, foreground, and background There is an entire body of research put into uncovering the topical structure of clauses and texts. There is a long tradition of semantic and pragmatic study of clause structure from the Charles University in Prague (e.g. Hajičová, 1993), there are several results supporting our hypotheses using the general theory of transitivity (Halliday, 1967, 1978; Hopper, 1979), there are numbers of algorithms for anaphor resolution which touch clausal categorization, there are studies of automatic summarization algorithms, and there are studies of text grammars which all have bearing on our work. However, no studies have been made specifically on clausal categorization for topical analysis, and the empirical validation of these ideas have been held back for lack of effective tools. Transitivity and clauses Transitivity is one of the most basic notions in the system of language, but ill formalized in the formal study of language. Copyright c 2002, American Association for Artificial Intelligence (www.aaai.org). All rights reserved. Clauses in language represent events and processes of various kinds, and transitivity is that characteristic of a clause which models the character of the process or event it represents. This systemic model was first formulated by Halliday (1967) and has since been elaborated by Hopper and others in a theoretic sense: very little empirical study on large numbers of texts has been performed, and no systematic let alone quantitative evaluation of the theories has even been proposed. One of the basic conceptual structures of language in use is that actions are done by people and affect things. How the action is performed, by whom, and on what are all encoded in the clause by various syntactic mechanisms, in a general system of transitivity. For most non-linguists, transitivity is only explicitly mentioned in foreign-language classes when classifying verbs as transitive or intransitive, meaning if the verb in question takes a direct object or prefers not to. This is of course central to the task of modeling action and effect, but transitivity covers more than this one aspect of process structure. Halliday’s model mentions a number of specific factors or “systems” that cover the more general “system” of transitivity: Number, type, and role of participant: human or not? Agent? Benefactive?; Process type: existence, possession, spatial/locative, spatial/mobile (e.g. 1978, p. 118). These aspects of clausal organization hook up with factors such as temporal, aspectual, or mood systems to produce a clause. This clause not only carries information about the event or process it represents, but it also crucially builds a text, together with adjacent clauses. In Halliday’s model (most comprehensively delineated in his 1967 publication) a clause is the confluence of three systems of syntactic choice: transitivity, mood and theme. Transitivity, he writes, is the set of options relating to cognitive content, mood being the system for organizing the utterance into a speech situation, and theme being the system for organizing the utterance into a discourse. While there is ample psycholinguistic evidence that the syntactic form of a clause is discarded after being processed by the hearer or reader (e.g. Jarvella, 1979), the communicative structure of the clause is retained to organize the information content of the text or discourse. The structure of a clause is not arbitrary, and cannot be determined in isolation from other clauses in the vicinity and other events, processes, and participants represented and mentioned in the Table 1: Transitivity characteristics.